In recent years, digital processing for images has been coming into wide use. As a method for compressively coding image data, high-efficiency coding such as MPEG (Moving Picture Experts Group) has been adopted. The high-efficiency coding is a technique for coding image data at a low bit rate so as to increase the efficiency of digital transmission, recording and the like.
In the current NTSC television broadcasts, an interlaced scanning method is adopted. According to the interlaced scanning method, one screen (one frame) is divided into a top field and a bottom field and is transmitted, thereby saving the bandwidth and enabling a high-efficiency transmission. However, line flicker and line crawling is conspicuous in the interlaced scanning method due to the high intensity and enlarged screen of the television receiver. Therefore, in some cases, deinterlacing for converting an interlaced image signal into a progressive image signal is performed by carrying out interpolation by using an image memory.
Japanese Published Patent Application No. Hei. 7-131761 (Document 1) discloses an example of a deinterlacing method for converting an interlaced image signal into a progressive image signal. This method is also called a VT filtering method. According to this VT filtering method, pixels in three successive fields, i.e., a field for which an interpolation pixel is generated and forward and backward fields, are subjected to a filtering process so as generate the interpolation pixel, and an interlaced image is converted into a progressive image by using the generated interpolation pixel. Hereinafter, this VT filtering method will be described with reference to FIG. 25.
FIG. 25 is a diagram schematically showing a state of an interlaced image on a time-vertical plane, in which a white dot shows a pixel (scan line). Here, a description is given of a filtering process in a case where a pixel in a position “k′” as shown in FIG. 25 is generated so as to perform interpolation.
In this case, by using the VT filtering method, pixels of three successive fields, i.e., a field including the position “k′” and forward and backward fields, for example 10 pixels “a′”–“j′” as shown in FIG. 25, are subjected to the filtering process, thereby generating the pixel of the position “k′”. At that time, a filter having vertical low-pass characteristics for pixels in the same field (pixels “d′”–“g′”) and vertical high-pass characteristics for pixels in adjacent fields (pixels “a′”–“c′” and “h′”–“j′”) is used in the filtering process. The filter having such characteristics extracts vertical low-pass components from the pixels “d′”–“g′” and vertical high-pass components from the pixels “a′”–“c′” and “h′”–“j′”. According to the VT filtering method, the low-pass components and high-pass components which are extracted by the filter are added so as to generate the interpolation pixel.
Japanese Published Patent Application No. Hei. 11-331782 (Document 2) discloses another example of the deinterlacing method. The method disclosed in this application is a method for implementing the deinterlacing method by a signal conversion apparatus having a still picture filter, a moving picture filter, a motion detection circuit and a mixer. According to this method, initially, the still picture filter and the motion picture filter respectively generate an interpolation pixel. Next, based on a result of detection by the motion detection circuit for detecting the motion of an image, the mixer mixes the interpolation signals that are outputted from the still picture filter and the moving picture filter so as to generate a final interpolation pixel. Then, by using the final interpolation pixel, the interlaced image is converted into a progressive image. Here, the motion detection circuit calculates a difference between frames and obtains the quantity of motion from the absolute value of the calculated difference.
Hereinafter, this deinterlacing method will be described with reference to FIG. 25. Here, a description is given of the filtering process in a case where a pixel in a position “k′” as shown in FIG. 25 is generated so as to perform interpolation.
According to this method, initially, the still picture filter subjects the pixels “b′”, “e′”, “f′” and “i′” to the filtering process so as to generate an interpolation pixel “k′”, and the moving picture filter subjects the pixels “a′”–“j′” to the filtering process so as to generate an interpolation pixel “k′”. Here, the filter coefficient of the still picture filter is set so that weights to the pixels “b′” and “i′” are the heaviest, and the filter coefficient of the moving picture filter is set so that weights to the pixels “d′”, “e′”, “f′” and “g′” are heavier. Next, the motion detection circuit detects a motion of the image from a difference between frames, and based on the detection result, the mixer mixes the interpolation signal that is outputted from the still picture filter and the interpolation signal that is outputted from the moving picture filter so as to generate the final interpolation pixel.
In the above description, the filtering process is performed for pixels in three fields, while the filtering process may be performed for pixels in two fields. In that case, the still picture filter subjects the pixels “e′”, “f′” and “i′” to the filtering process, and the moving picture filter subjects the pixels “d′”–“j′” to the filtering process, thereby generating the interpolation pixel “k′”.
As another example of the deinterlacing method, there is a method which detects a motion vector between a field that is subjected to the deinterlacing and its adjacent field, obtains an interpolation pixel from the adjacent field based on the detection result of the motion vector, and converts an interlaced image into a progressive image by using the interpolation pixel. According to this method, since the motion of the image is detected as the motion vector information, the interpolation process which almost matches the motion of the image can be performed, while the detection of the motion vector requires a massive quantity of signal processing.
Thus, Japanese Published Patent Application No. Hei. 10-126749 (Document 3) discloses a deinterlacing apparatus which implements a method for performing deinterlacing by using a motion vector, in which the quantity of processing for detecting the motion vector is reduced.
The deinterlacing apparatus disclosed in this application receives an interlaced image signal (MPEG video) that is coded by a coding method such as MPEG, and adaptively changes the interpolation method in the deinterlacing by utilizing information such as a structure of an image which is included in the MPEG video and a motion vector, whereby circuits such as a motion detection circuit and a motion vector detection circuit are omitted. Here, a description is given of a case where the MPEG video that is inputted to the deinterlacing apparatus is coded by a MPEG2 coding method.
The MPEG2 coding process includes an intra-frame coding and an inter-frame coding. When the intra-frame coding is performed to an interlaced image, the interlaced image is subjected to a DCT (Discrete Cosine Transformation) process, for example, in units of block (DCT block) being composed of 8×8 pixels, its spatial coordinate components are converted into frequency components, and thereafter, a variable-length coding is performed.
On the other hand, when the inter-frame coding is performed to the interlaced image, differences between the image of the present frame and reference images of forward and backward frames are obtained as predicted errors, and the predicted errors are subjected to a DCT process, a quantization process and a variable-length coding process. Accordingly, the quantity of codes can be significantly decreased. However, in a case where the motion of the image is large, when the differences between the present image and the forward and backward images are simply obtained, the predicted errors may be large, and therefore, the quantity of codes may be adversely increased. Then, motion vectors between the reference images and the image of the present frame are detected, and the reference images are motion-compensated based on the motion vectors so as to obtain the differences from the image of the present frame, and as a result, the predicted errors are reduced so as to decrease the quantity of codes. Here, in the MPEG2 coding, one image (picture) which consists of one frame or one field is divided into macroblocks, where each macroblock is composed of 16 lines×16 pixels, and then the motion vector is detected in units of macroblock, thereby performing the motion compensation.
Among images which have been coded in the above-mentioned coding process, an intra-frame coded image that is obtained by predictive-coding image data in one frame is called an I picture, an inter-frame forward predictive-coded image that is obtained by coding image data by a predictive coding using a forward frame is called a P picture, and a bidirectionally predictive-coded image that is obtained by coding image data by a predictive coding using any of a forward frame, a backward frame, and bidirectional frames is called a B picture.
The coding of the interlaced image is performed in the following cases: (1) a case where images of top and bottom fields are directly coded, i.e., the coding is performed in states of field structure, and (2) a case where a frame image is made from top and bottom field images and the coding is performed in a state of frame structure. In the coding that is performed in the state of the field structure, the motion compensation prediction and the DCT coding are performed in units of field picture. On the other hand, in the coding in the state of the frame structure, the coding is performed in units of frame picture, and frame prediction, which uses a frame image as a reference image, and field prediction, which uses one of the top field image and the bottom field image as a reference image, are adopted as the motion compensation prediction.
Therefore, by decoding a code sequence of MPEG video which is coded as described above, the deinterlacing apparatus can obtain information such as the image structure at the coding of an interlaced image, the motion vector and the prediction mode. Then, the deinterlacing apparatus judges whether the decoded image data is a static area or a moving area based on the obtained information. When the deinterlacing apparatus judges that the image data is a static area, the deinterlacing apparatus performs the inter-field interpolation, and when the deinterlacing apparatus judges that the image data is a moving area, the deinterlacing apparatus performs the intra-field interpolation, thereby converting the interlaced image signal into a progressive image signal.
However, in the deinterlacing, the vertical resolution of the progressive image can be increased more so than in the case when the inter-field interpolation is employed, as compared to the case where the intra-field interpolation is employed. Therefore, even when the decoded image data is a moving area, the deinterlacing apparatus judges whether the inter-field interpolation can be performed or not. Here, the quantity of motion of the motion vector at the coding which is included in the MPEG video is calculated, and when the quantity of motion is equivalent to even-numbered pixels, it is judged that the inter-field interpolation can be performed. Then, when it is judged that the inter-field interpolation can be performed even in a moving area, the inter-field interpolation is performed based on the motion vector, and as a result, the vertical resolution of the converted progressive image can be increased.
However, according to the VT filtering method described in Document 1, the interpolation pixel is generated regardless of whether the interlaced image is a still picture or a moving picture. That is, the filtering process is always performed to pixels of three successive fields, i.e., a field for which an interpolation pixel is generated and forward and backward fields. Thus, in the case of a still picture, a high-resolution progressive image can be generated. On the other hand, in the case of a moving picture, the correlation between the field for which the interpolation pixel is generated and the forward and backward fields is low, and thus, when pixels of the forward and backward fields are subjected to the filtering process to generate the interpolation pixel, degradation of the image quality may occur. Particularly in a case where video including an oblique edge moves vertically or the like, degradation occurs; for example, the edge looks jagged.
In addition, according to the method described in Document 2, two kinds of filters, i.e., the still picture filter and the moving picture filter, are required for generating an interpolation pixel, and as a result of requiring these two kinds of filters, the circuit scale is adversely increased.
Further, generally, according to the MPEG2 coding method, when an interlaced image is motion-compensated and then predictive-coded, a motion vector between frames which are one to three frames away from each other is detected, and a reference image is motion-compensated by using the motion vector. Therefore, when the deinterlacing apparatus performs the inter-field interpolation by directly using the motion vector that is obtained at the decoding of MPEG video, an interpolation pixel may be obtained from a frame that is several frames apart from the target frame. When a motion vector is used at the motion compensation in the coding, even when its detection accuracy is low, only the coding efficiency is slightly worsened and the image quality is not remarkably degraded. Therefore, there is no problem when the interpolation pixel is obtained from a frame that is several frames apart from the target frame. However, in the deinterlacing that performs interpolation of pixels based on a motion vector, the motion vector should be detected with a higher accuracy. Therefore, in the deinterlacing apparatus described in Document 3, when an interpolation pixel is obtained by directly using a motion vector at the motion compensation in the coding, the interpolation pixel may be obtained from a frame that is several frames apart from the target frame, and thus, an erroneous interpolation pixel may be generated.